{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ItXfxkxvosLH"
   },
   "source": [
    "# TensorFlow and TextAttack"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yVOBJvgvjt_M"
   },
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/QData/TextAttack/blob/master/docs/2notebook/Example_0_tensorflow.ipynb)\n",
    "\n",
    "[![View Source on GitHub](https://img.shields.io/badge/github-view%20source-black.svg)](https://github.com/QData/TextAttack/blob/master/docs/2notebook/Example_0_tensorflow.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WooZ9pGnNJbv"
   },
   "source": [
    "Please remember to run  **pip3 install textattack[tensorflow]**  in your notebook enviroment before the following codes:\n",
    "\n",
    "## Run textattack on a trained tensorflow model: \n",
    "\n",
    "### First: Training\n",
    "\n",
    "The following is code for training a text classification model using TensorFlow (and on top of it, the Keras API). This comes from the Tensorflow documentation ([see here](https://www.tensorflow.org/tutorials/keras/text_classification_with_hub)).\n",
    "\n",
    "This cell loads the IMDB dataset (using `tensorflow_datasets`, not `datasets`), initializes a simple classifier, and trains it using Keras."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000,
     "referenced_widgets": [
      "d5599f263bb24eb18b3d2c308703d53a",
      "82089574f05e40599c476b100bd4df3e",
      "36772b07b54f41549989cf91a71298d3",
      "e70535ff314f45a1aaa82701767ec34e",
      "a4b26cbdbbda4a599c4e393c8c5a65c2",
      "bf37d9a3bdb54068a836300727297311",
      "37080c38307c49b4b693e47530eb3f79",
      "06520a28f4d0484b80d943e3624e71f9",
      "e8334ca8eec146a88754e8738f881bfe",
      "9a8edbd9aabe4c4a81480aada860527c",
      "46535add886e425095999ac0944dba1f",
      "eaf889342ad54eeca243a1cad2127709",
      "1f42009b89fb484ea9d2eb5a741a29f5",
      "3042da9f7c7e45f48ca3ca84ebf13feb",
      "4774670879ab4255934fed8bfd524b53",
      "afa8c0699dc54bbe954899dd4a16925a",
      "1b0d3c901e044b87aa7e26113ad21ba9",
      "2115144afd7642d6907f25369ee81420",
      "14ea179e0cdf42d7a19bdc7b06152acf",
      "adb12cf12e3e45d8923f9a152483b545",
      "a885fdb51d7a4bce92a07b34d4a38ba1",
      "2160b7d916374d74a2b1b7e748e741d9",
      "bde934bd43524c5cbd018939cde974c0",
      "2b7ff7d6781e4dfda2acca55147faab8",
      "dc8f7d3995d541b4bd6a38f1dedfdbcf",
      "30227a5c0ec547418592c302c068fbea",
      "cfb4bad9707f48749826f6f4893892ae",
      "a0bc83d3fcc242ac80eed93d8b08dcf4",
      "2eec1883fe3b49cf89279f5274291a8f",
      "a61b2731e28f4c63add42337e2312270",
      "d463260a18ea43f9924b07e4fc75db68",
      "838e89b5868d413996a578e1c72375cd",
      "7fea1b30c021405cb9fbc62478a50f98",
      "14033e9b43cf4fd58292f8c1e4b00f9a",
      "65965b0834054e3fa55d773939a10c58",
      "598169b8b8a24139852c58def2084930",
      "3a74d622a15e48adb392881451783d55",
      "bbb5981bc2664412b67bb5adf99e7ae2",
      "5441622e6eb34e0ba3bd3bffb97e6d19",
      "79b0d042a4424a1a84da8d6f712e961e",
      "138bcfe95c97485b97b610795a316c64",
      "99b551174e8b45c789071f7fa158b6db",
      "da0a56d359b84ee4a3ac8d74e8664c74",
      "f448086e52b247fb96a2baa83fec699c",
      "6354a1c2c0a14f289205492e3f1cd8ff",
      "883ef3211ea54da1bfa20ce16fd7dc64",
      "ce80fa9b8bcc4d068aec4e793e88dd26",
      "ece503519fa94bad9f67af5e77370442",
      "64d3051caacd43d0b492ddea27bf19c7",
      "c665d7c79d2d4663987bf37822f1bbd8",
      "efa3f49065044069870b26a6b94ad57c",
      "639ec2f0e0994b2c92a6faf25fb1b72c",
      "c119fbc07fdf4c92af1ff62867335560",
      "d7172567bc6443808b945ea59dcbdee1",
      "f63c0c5e78d741ce9c69b7461f91168c",
      "48b3d69a160247659f5b66de5bb3f1b8",
      "9bc65ec690e14edbbae65cae681dd7f8",
      "9a128749e0184172baa45c5dab27fdc7",
      "02b16464f92b4a9abffc30238743d890",
      "c308c4e969834728ab9e4043ecc18bfb",
      "bb00bb5c6e9b49cd8d4c959b4c1cc7a3",
      "21b0fd860d314fd6a27435c405b73df2",
      "6e663cf589c041418c946a00e0a2fca2",
      "d4f8e73be99e4efcb7bc94ea9e3dbffb"
     ]
    },
    "id": "2ew7HTbPpCJH",
    "outputId": "c8f41922-13b7-4aea-d263-496025a1725a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Version:  2.3.2\n",
      "Eager mode:  True\n",
      "Hub version:  0.12.0\n",
      "GPU is NOT AVAILABLE\n",
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "keras_layer (KerasLayer)     (None, 20)                400020    \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 16)                336       \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 17        \n",
      "=================================================================\n",
      "Total params: 400,373\n",
      "Trainable params: 400,373\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Epoch 1/40\n",
      "30/30 [==============================] - 2s 60ms/step - loss: 1.1498 - accuracy: 0.5131 - val_loss: 0.7321 - val_accuracy: 0.5698\n",
      "Epoch 2/40\n",
      "30/30 [==============================] - 1s 39ms/step - loss: 0.6914 - accuracy: 0.5994 - val_loss: 0.6636 - val_accuracy: 0.6008\n",
      "Epoch 3/40\n",
      "30/30 [==============================] - 1s 37ms/step - loss: 0.6423 - accuracy: 0.6219 - val_loss: 0.6330 - val_accuracy: 0.6367\n",
      "Epoch 4/40\n",
      "30/30 [==============================] - 1s 37ms/step - loss: 0.6107 - accuracy: 0.6524 - val_loss: 0.6043 - val_accuracy: 0.6602\n",
      "Epoch 5/40\n",
      "30/30 [==============================] - 1s 38ms/step - loss: 0.5759 - accuracy: 0.6832 - val_loss: 0.5736 - val_accuracy: 0.6943\n",
      "Epoch 6/40\n",
      "30/30 [==============================] - 1s 38ms/step - loss: 0.5390 - accuracy: 0.7149 - val_loss: 0.5391 - val_accuracy: 0.7141\n",
      "Epoch 7/40\n",
      "30/30 [==============================] - 1s 37ms/step - loss: 0.5004 - accuracy: 0.7480 - val_loss: 0.5068 - val_accuracy: 0.7393\n",
      "Epoch 8/40\n",
      "30/30 [==============================] - 1s 43ms/step - loss: 0.4632 - accuracy: 0.7733 - val_loss: 0.4773 - val_accuracy: 0.7634\n",
      "Epoch 9/40\n",
      "30/30 [==============================] - 1s 44ms/step - loss: 0.4292 - accuracy: 0.7986 - val_loss: 0.4502 - val_accuracy: 0.7739\n",
      "Epoch 10/40\n",
      "30/30 [==============================] - 1s 43ms/step - loss: 0.3963 - accuracy: 0.8205 - val_loss: 0.4271 - val_accuracy: 0.7863\n",
      "Epoch 11/40\n",
      "30/30 [==============================] - 1s 41ms/step - loss: 0.3664 - accuracy: 0.8357 - val_loss: 0.4048 - val_accuracy: 0.8109\n",
      "Epoch 12/40\n",
      "30/30 [==============================] - 1s 40ms/step - loss: 0.3396 - accuracy: 0.8533 - val_loss: 0.3864 - val_accuracy: 0.8146\n",
      "Epoch 13/40\n",
      "30/30 [==============================] - 1s 42ms/step - loss: 0.3147 - accuracy: 0.8681 - val_loss: 0.3699 - val_accuracy: 0.8347\n",
      "Epoch 14/40\n",
      "30/30 [==============================] - 1s 38ms/step - loss: 0.2918 - accuracy: 0.8788 - val_loss: 0.3563 - val_accuracy: 0.8356\n",
      "Epoch 15/40\n",
      "30/30 [==============================] - 1s 43ms/step - loss: 0.2717 - accuracy: 0.8881 - val_loss: 0.3452 - val_accuracy: 0.8393\n",
      "Epoch 16/40\n",
      "30/30 [==============================] - 1s 43ms/step - loss: 0.2540 - accuracy: 0.8965 - val_loss: 0.3347 - val_accuracy: 0.8493\n",
      "Epoch 17/40\n",
      "30/30 [==============================] - 1s 40ms/step - loss: 0.2379 - accuracy: 0.9045 - val_loss: 0.3272 - val_accuracy: 0.8533\n",
      "Epoch 18/40\n",
      "30/30 [==============================] - 1s 46ms/step - loss: 0.2230 - accuracy: 0.9114 - val_loss: 0.3204 - val_accuracy: 0.8616\n",
      "Epoch 19/40\n",
      "30/30 [==============================] - 1s 47ms/step - loss: 0.2103 - accuracy: 0.9177 - val_loss: 0.3181 - val_accuracy: 0.8562\n",
      "Epoch 20/40\n",
      "30/30 [==============================] - 1s 45ms/step - loss: 0.1977 - accuracy: 0.9243 - val_loss: 0.3119 - val_accuracy: 0.8660\n",
      "Epoch 21/40\n",
      "30/30 [==============================] - 1s 38ms/step - loss: 0.1861 - accuracy: 0.9303 - val_loss: 0.3093 - val_accuracy: 0.8729\n",
      "Epoch 22/40\n",
      "30/30 [==============================] - 1s 37ms/step - loss: 0.1759 - accuracy: 0.9337 - val_loss: 0.3075 - val_accuracy: 0.8704\n",
      "Epoch 23/40\n",
      "30/30 [==============================] - 1s 37ms/step - loss: 0.1661 - accuracy: 0.9393 - val_loss: 0.3061 - val_accuracy: 0.8719\n",
      "Epoch 24/40\n",
      "30/30 [==============================] - 1s 40ms/step - loss: 0.1564 - accuracy: 0.9439 - val_loss: 0.3077 - val_accuracy: 0.8745\n",
      "Epoch 25/40\n",
      "30/30 [==============================] - 2s 51ms/step - loss: 0.1461 - accuracy: 0.9475 - val_loss: 0.3077 - val_accuracy: 0.8739\n",
      "Epoch 26/40\n",
      "30/30 [==============================] - 1s 47ms/step - loss: 0.1363 - accuracy: 0.9524 - val_loss: 0.3098 - val_accuracy: 0.8714\n",
      "Epoch 27/40\n",
      "30/30 [==============================] - 1s 40ms/step - loss: 0.1283 - accuracy: 0.9551 - val_loss: 0.3113 - val_accuracy: 0.8727\n",
      "Epoch 28/40\n",
      "30/30 [==============================] - 1s 41ms/step - loss: 0.1226 - accuracy: 0.9577 - val_loss: 0.3142 - val_accuracy: 0.8746\n",
      "Epoch 29/40\n",
      "30/30 [==============================] - 1s 36ms/step - loss: 0.1132 - accuracy: 0.9623 - val_loss: 0.3166 - val_accuracy: 0.8733\n",
      "Epoch 30/40\n",
      "30/30 [==============================] - 1s 39ms/step - loss: 0.1059 - accuracy: 0.9661 - val_loss: 0.3207 - val_accuracy: 0.8699\n",
      "Epoch 31/40\n",
      "30/30 [==============================] - 1s 38ms/step - loss: 0.0994 - accuracy: 0.9683 - val_loss: 0.3240 - val_accuracy: 0.8692\n",
      "Epoch 32/40\n",
      "30/30 [==============================] - 1s 38ms/step - loss: 0.0945 - accuracy: 0.9711 - val_loss: 0.3285 - val_accuracy: 0.8687\n",
      "Epoch 33/40\n",
      "30/30 [==============================] - 1s 36ms/step - loss: 0.0877 - accuracy: 0.9744 - val_loss: 0.3327 - val_accuracy: 0.8694\n",
      "Epoch 34/40\n",
      "30/30 [==============================] - 1s 40ms/step - loss: 0.0823 - accuracy: 0.9761 - val_loss: 0.3414 - val_accuracy: 0.8658\n",
      "Epoch 35/40\n",
      "30/30 [==============================] - 1s 39ms/step - loss: 0.0773 - accuracy: 0.9785 - val_loss: 0.3423 - val_accuracy: 0.8712\n",
      "Epoch 36/40\n",
      "30/30 [==============================] - 1s 36ms/step - loss: 0.0723 - accuracy: 0.9805 - val_loss: 0.3481 - val_accuracy: 0.8674\n",
      "Epoch 37/40\n",
      "30/30 [==============================] - 1s 37ms/step - loss: 0.0681 - accuracy: 0.9823 - val_loss: 0.3597 - val_accuracy: 0.8624\n",
      "Epoch 38/40\n",
      "30/30 [==============================] - 1s 37ms/step - loss: 0.0639 - accuracy: 0.9831 - val_loss: 0.3589 - val_accuracy: 0.8691\n",
      "Epoch 39/40\n",
      "30/30 [==============================] - 1s 42ms/step - loss: 0.0595 - accuracy: 0.9857 - val_loss: 0.3650 - val_accuracy: 0.8720\n",
      "Epoch 40/40\n",
      "30/30 [==============================] - 1s 43ms/step - loss: 0.0558 - accuracy: 0.9868 - val_loss: 0.3711 - val_accuracy: 0.8696\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "import tensorflow as tf\n",
    "import tensorflow_hub as hub\n",
    "import tensorflow_datasets as tfds\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "print(\"Version: \", tf.__version__)\n",
    "print(\"Eager mode: \", tf.executing_eagerly())\n",
    "print(\"Hub version: \", hub.__version__)\n",
    "print(\n",
    "    \"GPU is\", \"available\" if tf.config.list_physical_devices(\"GPU\") else \"NOT AVAILABLE\"\n",
    ")\n",
    "\n",
    "train_data, test_data = tfds.load(\n",
    "    name=\"imdb_reviews\", split=[\"train\", \"test\"], batch_size=-1, as_supervised=True\n",
    ")\n",
    "\n",
    "train_examples, train_labels = tfds.as_numpy(train_data)\n",
    "test_examples, test_labels = tfds.as_numpy(test_data)\n",
    "\n",
    "model = \"https://tfhub.dev/google/tf2-preview/gnews-swivel-20dim/1\"\n",
    "hub_layer = hub.KerasLayer(\n",
    "    model, output_shape=[20], input_shape=[], dtype=tf.string, trainable=True\n",
    ")\n",
    "hub_layer(train_examples[:3])\n",
    "\n",
    "model = tf.keras.Sequential()\n",
    "model.add(hub_layer)\n",
    "model.add(tf.keras.layers.Dense(16, activation=\"relu\"))\n",
    "model.add(tf.keras.layers.Dense(1))\n",
    "\n",
    "model.summary()\n",
    "\n",
    "x_val = train_examples[:10000]\n",
    "partial_x_train = train_examples[10000:]\n",
    "\n",
    "y_val = train_labels[:10000]\n",
    "partial_y_train = train_labels[10000:]\n",
    "\n",
    "model.compile(\n",
    "    optimizer=\"adam\",\n",
    "    loss=tf.losses.BinaryCrossentropy(from_logits=True),\n",
    "    metrics=[\"accuracy\"],\n",
    ")\n",
    "\n",
    "history = model.fit(\n",
    "    partial_x_train,\n",
    "    partial_y_train,\n",
    "    epochs=40,\n",
    "    batch_size=512,\n",
    "    validation_data=(x_val, y_val),\n",
    "    verbose=1,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3varlQvrnHqV"
   },
   "source": [
    "### Attacking\n",
    "\n",
    "For each input, our classifier outputs a single number that indicates how positive or negative the model finds the input. For binary classification, TextAttack expects two numbers for each input (a score for each class, positive and negative). We have to post-process each output to fit this TextAttack format. To add this post-processing we need to implement a custom model wrapper class (instead of using the built-in `textattack.models.wrappers.TensorFlowModelWrapper`).\n",
    "\n",
    "Each `ModelWrapper` must implement a single method, `__call__`, which takes a list of strings and returns a `List`, `np.ndarray`, or `torch.Tensor` of predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fHX3Lo7wU2LM",
    "outputId": "aa98c5c8-c74c-4e07-d3cc-6e8f26fbd7ca"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import torch\n",
    "\n",
    "from textattack.models.wrappers import ModelWrapper\n",
    "\n",
    "\n",
    "class CustomTensorFlowModelWrapper(ModelWrapper):\n",
    "    def __init__(self, model):\n",
    "        self.model = model\n",
    "\n",
    "    def __call__(self, text_input_list):\n",
    "        text_array = np.array(text_input_list)\n",
    "        preds = self.model(text_array).numpy()\n",
    "        logits = torch.exp(-torch.tensor(preds))\n",
    "        logits = 1 / (1 + logits)\n",
    "        logits = logits.squeeze(dim=-1)\n",
    "        # Since this model only has a single output (between 0 or 1),\n",
    "        # we have to add the second dimension.\n",
    "        final_preds = torch.stack((1 - logits, logits), dim=1)\n",
    "        return final_preds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ku71HuZ4n7ih"
   },
   "source": [
    "Let's test our model wrapper out to make sure it can use our model to return predictions in the correct format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "id": "9hgiLQC4ejmM",
    "outputId": "132c3be5-fe5e-4be4-ef98-5c2efedc0dfd"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.1409, 0.8591],\n",
       "        [0.0213, 0.9787]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "CustomTensorFlowModelWrapper(model)([\"I hate you so much\", \"I love you\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-Bs14Hr4n_Sp"
   },
   "source": [
    "Looks good! Now we can initialize our model wrapper with the model we trained and pass it to an instance of `textattack.attack.Attack`. \n",
    "\n",
    "We'll use the `PWWSRen2019` recipe as our attack, and attack 10 samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "07mOE-wLVQDR",
    "outputId": "0b7e23de-2dc0-43ab-8e58-3863e112b159"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:datasets.builder:Using custom data configuration default\n",
      "WARNING:datasets.builder:Reusing dataset rotten_tomatoes_movie_review (/p/qdata/jy2ma/.cache/textattack/datasets/rotten_tomatoes_movie_review/default/1.0.0/9c411f7ecd9f3045389de0d9ce984061a1056507703d2e3183b1ac1a90816e4d)\n",
      "textattack: Loading \u001b[94mdatasets\u001b[0m dataset \u001b[94mrotten_tomatoes\u001b[0m, split \u001b[94mtest\u001b[0m.\n",
      "textattack: Unknown if model of class <class 'tensorflow.python.keras.engine.sequential.Sequential'> compatible with goal function <class 'textattack.goal_functions.classification.untargeted_classification.UntargetedClassification'>.\n",
      "[Succeeded / Failed / Skipped / Total] 2 / 0 / 3 / 5:  50%|█████     | 5/10 [00:00<00:00, 43.42it/s] "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attack(\n",
      "  (search_method): GreedyWordSwapWIR(\n",
      "    (wir_method):  weighted-saliency\n",
      "  )\n",
      "  (goal_function):  UntargetedClassification\n",
      "  (transformation):  WordSwapWordNet\n",
      "  (constraints): \n",
      "    (0): RepeatModification\n",
      "    (1): StopwordModification\n",
      "  (is_black_box):  True\n",
      ") \n",
      "\n",
      "--------------------------------------------- Result 1 ---------------------------------------------\n",
      "\u001b[91mNegative (90%)\u001b[0m --> \u001b[37m[SKIPPED]\u001b[0m\n",
      "\n",
      "lovingly photographed in the manner of a golden book sprung to life , stuart little 2 manages sweetness largely without stickiness .\n",
      "\n",
      "\n",
      "--------------------------------------------- Result 2 ---------------------------------------------\n",
      "\u001b[92mPositive (52%)\u001b[0m --> \u001b[91mNegative (97%)\u001b[0m\n",
      "\n",
      "consistently clever and \u001b[92msuspenseful\u001b[0m .\n",
      "\n",
      "consistently clever and \u001b[91mcliff-hanging\u001b[0m .\n",
      "\n",
      "\n",
      "--------------------------------------------- Result 3 ---------------------------------------------\n",
      "\u001b[92mPositive (89%)\u001b[0m --> \u001b[91mNegative (86%)\u001b[0m\n",
      "\n",
      "it's like a \" big chill \" reunion of the baader-meinhof gang , only these \u001b[92mguys\u001b[0m are more harmless pranksters than political activists .\n",
      "\n",
      "it's like a \" big chill \" reunion of the baader-meinhof gang , only these \u001b[91mroast\u001b[0m are more harmless pranksters than political activists .\n",
      "\n",
      "\n",
      "--------------------------------------------- Result 4 ---------------------------------------------\n",
      "\u001b[91mNegative (60%)\u001b[0m --> \u001b[37m[SKIPPED]\u001b[0m\n",
      "\n",
      "the story gives ample opportunity for large-scale action and suspense , which director shekhar kapur supplies with tremendous skill .\n",
      "\n",
      "\n",
      "--------------------------------------------- Result 5 ---------------------------------------------\n",
      "\u001b[91mNegative (99%)\u001b[0m --> \u001b[37m[SKIPPED]\u001b[0m\n",
      "\n",
      "red dragon \" never cuts corners .\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Succeeded / Failed / Skipped / Total] 4 / 0 / 3 / 7:  70%|███████   | 7/10 [00:00<00:00, 18.97it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 6 ---------------------------------------------\n",
      "\u001b[92mPositive (99%)\u001b[0m --> \u001b[91mNegative (85%)\u001b[0m\n",
      "\n",
      "fresnadillo has something serious to say about the \u001b[92mways\u001b[0m in which \u001b[92mextravagant\u001b[0m chance can distort our perspective and throw us off the path of \u001b[92mgood\u001b[0m sense .\n",
      "\n",
      "fresnadillo has something serious to say about the \u001b[91mmanner\u001b[0m in which \u001b[91mexuberant\u001b[0m chance can distort our perspective and throw us off the path of \u001b[91mripe\u001b[0m sense .\n",
      "\n",
      "\n",
      "--------------------------------------------- Result 7 ---------------------------------------------\n",
      "\u001b[92mPositive (99%)\u001b[0m --> \u001b[91mNegative (73%)\u001b[0m\n",
      "\n",
      "\u001b[92mthrows\u001b[0m in enough clever and unexpected \u001b[92mtwists\u001b[0m to make the formula feel fresh .\n",
      "\n",
      "\u001b[91mflip\u001b[0m in enough clever and unexpected \u001b[91mconstruction\u001b[0m to make the formula feel fresh .\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Succeeded / Failed / Skipped / Total] 6 / 0 / 4 / 10: 100%|██████████| 10/10 [00:00<00:00, 17.90it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 8 ---------------------------------------------\n",
      "\u001b[92mPositive (96%)\u001b[0m --> \u001b[91mNegative (93%)\u001b[0m\n",
      "\n",
      "weighty and ponderous but every \u001b[92mbit\u001b[0m as filling as the \u001b[92mtreat\u001b[0m of the title .\n",
      "\n",
      "weighty and ponderous but every \u001b[91mbite\u001b[0m as filling as the \u001b[91mcover\u001b[0m of the title .\n",
      "\n",
      "\n",
      "--------------------------------------------- Result 9 ---------------------------------------------\n",
      "\u001b[92mPositive (84%)\u001b[0m --> \u001b[91mNegative (70%)\u001b[0m\n",
      "\n",
      "a \u001b[92mreal\u001b[0m audience-pleaser that will strike a chord with anyone who's ever waited in a doctor's office , emergency room , hospital \u001b[92mbed\u001b[0m or insurance company office .\n",
      "\n",
      "a \u001b[91mmaterial\u001b[0m audience-pleaser that will strike a chord with anyone who's ever waited in a doctor's office , emergency room , hospital \u001b[91mscrew\u001b[0m or insurance company office .\n",
      "\n",
      "\n",
      "--------------------------------------------- Result 10 ---------------------------------------------\n",
      "\u001b[91mNegative (99%)\u001b[0m --> \u001b[37m[SKIPPED]\u001b[0m\n",
      "\n",
      "generates an enormous feeling of empathy for its characters .\n",
      "\n",
      "\n",
      "\n",
      "+-------------------------------+--------+\n",
      "| Attack Results                |        |\n",
      "+-------------------------------+--------+\n",
      "| Number of successful attacks: | 6      |\n",
      "| Number of failed attacks:     | 0      |\n",
      "| Number of skipped attacks:    | 4      |\n",
      "| Original accuracy:            | 60.0%  |\n",
      "| Accuracy under attack:        | 0.0%   |\n",
      "| Attack success rate:          | 100.0% |\n",
      "| Average perturbed word %:     | 13.2%  |\n",
      "| Average num. words per input: | 15.4   |\n",
      "| Avg num queries:              | 139.0  |\n",
      "+-------------------------------+--------+\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<textattack.attack_results.skipped_attack_result.SkippedAttackResult at 0x7f74758a3e50>,\n",
       " <textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7f74758a3c70>,\n",
       " <textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7f74758a3490>,\n",
       " <textattack.attack_results.skipped_attack_result.SkippedAttackResult at 0x7f74758a3fd0>,\n",
       " <textattack.attack_results.skipped_attack_result.SkippedAttackResult at 0x7f74758a39d0>,\n",
       " <textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7f7475903100>,\n",
       " <textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7f7475284460>,\n",
       " <textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7f74758a3910>,\n",
       " <textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7f74758a3790>,\n",
       " <textattack.attack_results.skipped_attack_result.SkippedAttackResult at 0x7f7489297c40>]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_wrapper = CustomTensorFlowModelWrapper(model)\n",
    "\n",
    "from textattack.datasets import HuggingFaceDataset\n",
    "from textattack.attack_recipes import PWWSRen2019\n",
    "from textattack import Attacker\n",
    "\n",
    "dataset = HuggingFaceDataset(\"rotten_tomatoes\", None, \"test\", shuffle=True)\n",
    "attack = PWWSRen2019.build(model_wrapper)\n",
    "\n",
    "attacker = Attacker(attack, dataset)\n",
    "attacker.attack_dataset()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "P3L9ccqGoS-J"
   },
   "source": [
    "## Conclusion \n",
    "\n",
    "Looks good! We successfully loaded a model, adapted it for TextAttack's `ModelWrapper`, and used that object in an attack. This is basically how you would adapt any model, using TensorFlow or any other library, for use with TextAttack."
   ]
  }
 ],
 "metadata": {
  "accelerator": "TPU",
  "colab": {
   "collapsed_sections": [],
   "name": "[TextAttack] tensorflow/keras example",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "02b16464f92b4a9abffc30238743d890": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "danger",
      "description": " 76%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_21b0fd860d314fd6a27435c405b73df2",
      "max": 50000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_bb00bb5c6e9b49cd8d4c959b4c1cc7a3",
      "value": 38047
     }
    },
    "06520a28f4d0484b80d943e3624e71f9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "138bcfe95c97485b97b610795a316c64": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_da0a56d359b84ee4a3ac8d74e8664c74",
       "IPY_MODEL_f448086e52b247fb96a2baa83fec699c"
      ],
      "layout": "IPY_MODEL_99b551174e8b45c789071f7fa158b6db"
     }
    },
    "14033e9b43cf4fd58292f8c1e4b00f9a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "14ea179e0cdf42d7a19bdc7b06152acf": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "info",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2160b7d916374d74a2b1b7e748e741d9",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_a885fdb51d7a4bce92a07b34d4a38ba1",
      "value": 1
     }
    },
    "1b0d3c901e044b87aa7e26113ad21ba9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_14ea179e0cdf42d7a19bdc7b06152acf",
       "IPY_MODEL_adb12cf12e3e45d8923f9a152483b545"
      ],
      "layout": "IPY_MODEL_2115144afd7642d6907f25369ee81420"
     }
    },
    "1f42009b89fb484ea9d2eb5a741a29f5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "2115144afd7642d6907f25369ee81420": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2160b7d916374d74a2b1b7e748e741d9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "21b0fd860d314fd6a27435c405b73df2": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2b7ff7d6781e4dfda2acca55147faab8": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2eec1883fe3b49cf89279f5274291a8f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "30227a5c0ec547418592c302c068fbea": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3042da9f7c7e45f48ca3ca84ebf13feb": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "36772b07b54f41549989cf91a71298d3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Dl Completed...: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bf37d9a3bdb54068a836300727297311",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_a4b26cbdbbda4a599c4e393c8c5a65c2",
      "value": 1
     }
    },
    "37080c38307c49b4b693e47530eb3f79": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3a74d622a15e48adb392881451783d55": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "46535add886e425095999ac0944dba1f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Dl Size...: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3042da9f7c7e45f48ca3ca84ebf13feb",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1f42009b89fb484ea9d2eb5a741a29f5",
      "value": 1
     }
    },
    "4774670879ab4255934fed8bfd524b53": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "48b3d69a160247659f5b66de5bb3f1b8": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5441622e6eb34e0ba3bd3bffb97e6d19": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "598169b8b8a24139852c58def2084930": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_79b0d042a4424a1a84da8d6f712e961e",
      "placeholder": "​",
      "style": "IPY_MODEL_5441622e6eb34e0ba3bd3bffb97e6d19",
      "value": " 25000/0 [00:14&lt;00:00, 3046.42 examples/s]"
     }
    },
    "6354a1c2c0a14f289205492e3f1cd8ff": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "639ec2f0e0994b2c92a6faf25fb1b72c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_48b3d69a160247659f5b66de5bb3f1b8",
      "placeholder": "​",
      "style": "IPY_MODEL_f63c0c5e78d741ce9c69b7461f91168c",
      "value": " 50000/0 [00:20&lt;00:00, 2774.64 examples/s]"
     }
    },
    "64d3051caacd43d0b492ddea27bf19c7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_efa3f49065044069870b26a6b94ad57c",
       "IPY_MODEL_639ec2f0e0994b2c92a6faf25fb1b72c"
      ],
      "layout": "IPY_MODEL_c665d7c79d2d4663987bf37822f1bbd8"
     }
    },
    "65965b0834054e3fa55d773939a10c58": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "info",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bbb5981bc2664412b67bb5adf99e7ae2",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_3a74d622a15e48adb392881451783d55",
      "value": 1
     }
    },
    "6e663cf589c041418c946a00e0a2fca2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "79b0d042a4424a1a84da8d6f712e961e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7fea1b30c021405cb9fbc62478a50f98": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_65965b0834054e3fa55d773939a10c58",
       "IPY_MODEL_598169b8b8a24139852c58def2084930"
      ],
      "layout": "IPY_MODEL_14033e9b43cf4fd58292f8c1e4b00f9a"
     }
    },
    "82089574f05e40599c476b100bd4df3e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "838e89b5868d413996a578e1c72375cd": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "883ef3211ea54da1bfa20ce16fd7dc64": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "99b551174e8b45c789071f7fa158b6db": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9a128749e0184172baa45c5dab27fdc7": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9a8edbd9aabe4c4a81480aada860527c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9bc65ec690e14edbbae65cae681dd7f8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_02b16464f92b4a9abffc30238743d890",
       "IPY_MODEL_c308c4e969834728ab9e4043ecc18bfb"
      ],
      "layout": "IPY_MODEL_9a128749e0184172baa45c5dab27fdc7"
     }
    },
    "a0bc83d3fcc242ac80eed93d8b08dcf4": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_838e89b5868d413996a578e1c72375cd",
      "placeholder": "​",
      "style": "IPY_MODEL_d463260a18ea43f9924b07e4fc75db68",
      "value": " 6690/25000 [00:00&lt;00:00, 66899.59 examples/s]"
     }
    },
    "a4b26cbdbbda4a599c4e393c8c5a65c2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "a61b2731e28f4c63add42337e2312270": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a885fdb51d7a4bce92a07b34d4a38ba1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "adb12cf12e3e45d8923f9a152483b545": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2b7ff7d6781e4dfda2acca55147faab8",
      "placeholder": "​",
      "style": "IPY_MODEL_bde934bd43524c5cbd018939cde974c0",
      "value": " 25000/0 [00:14&lt;00:00, 2939.35 examples/s]"
     }
    },
    "afa8c0699dc54bbe954899dd4a16925a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bb00bb5c6e9b49cd8d4c959b4c1cc7a3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "bbb5981bc2664412b67bb5adf99e7ae2": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bde934bd43524c5cbd018939cde974c0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "bf37d9a3bdb54068a836300727297311": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c119fbc07fdf4c92af1ff62867335560": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "c308c4e969834728ab9e4043ecc18bfb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d4f8e73be99e4efcb7bc94ea9e3dbffb",
      "placeholder": "​",
      "style": "IPY_MODEL_6e663cf589c041418c946a00e0a2fca2",
      "value": " 38047/50000 [00:00&lt;13:25, 14.84 examples/s]"
     }
    },
    "c665d7c79d2d4663987bf37822f1bbd8": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ce80fa9b8bcc4d068aec4e793e88dd26": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "cfb4bad9707f48749826f6f4893892ae": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "danger",
      "description": " 27%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a61b2731e28f4c63add42337e2312270",
      "max": 25000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2eec1883fe3b49cf89279f5274291a8f",
      "value": 6690
     }
    },
    "d463260a18ea43f9924b07e4fc75db68": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d4f8e73be99e4efcb7bc94ea9e3dbffb": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d5599f263bb24eb18b3d2c308703d53a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_36772b07b54f41549989cf91a71298d3",
       "IPY_MODEL_e70535ff314f45a1aaa82701767ec34e"
      ],
      "layout": "IPY_MODEL_82089574f05e40599c476b100bd4df3e"
     }
    },
    "d7172567bc6443808b945ea59dcbdee1": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "da0a56d359b84ee4a3ac8d74e8664c74": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "danger",
      "description": " 25%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_883ef3211ea54da1bfa20ce16fd7dc64",
      "max": 25000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_6354a1c2c0a14f289205492e3f1cd8ff",
      "value": 6178
     }
    },
    "dc8f7d3995d541b4bd6a38f1dedfdbcf": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_cfb4bad9707f48749826f6f4893892ae",
       "IPY_MODEL_a0bc83d3fcc242ac80eed93d8b08dcf4"
      ],
      "layout": "IPY_MODEL_30227a5c0ec547418592c302c068fbea"
     }
    },
    "e70535ff314f45a1aaa82701767ec34e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_06520a28f4d0484b80d943e3624e71f9",
      "placeholder": "​",
      "style": "IPY_MODEL_37080c38307c49b4b693e47530eb3f79",
      "value": " 1/1 [00:04&lt;00:00,  4.09s/ url]"
     }
    },
    "e8334ca8eec146a88754e8738f881bfe": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_46535add886e425095999ac0944dba1f",
       "IPY_MODEL_eaf889342ad54eeca243a1cad2127709"
      ],
      "layout": "IPY_MODEL_9a8edbd9aabe4c4a81480aada860527c"
     }
    },
    "eaf889342ad54eeca243a1cad2127709": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_afa8c0699dc54bbe954899dd4a16925a",
      "placeholder": "​",
      "style": "IPY_MODEL_4774670879ab4255934fed8bfd524b53",
      "value": " 80/80 [00:04&lt;00:00, 19.80 MiB/s]"
     }
    },
    "ece503519fa94bad9f67af5e77370442": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "efa3f49065044069870b26a6b94ad57c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "info",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d7172567bc6443808b945ea59dcbdee1",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_c119fbc07fdf4c92af1ff62867335560",
      "value": 1
     }
    },
    "f448086e52b247fb96a2baa83fec699c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ece503519fa94bad9f67af5e77370442",
      "placeholder": "​",
      "style": "IPY_MODEL_ce80fa9b8bcc4d068aec4e793e88dd26",
      "value": " 6178/25000 [00:00&lt;00:00, 61776.08 examples/s]"
     }
    },
    "f63c0c5e78d741ce9c69b7461f91168c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
